Gesture-based selection and manipulation method

ABSTRACT

A method for selecting multiple content items in a list of content items is presented. In an embodiment, the method first displays a list of content items along the first direction on a display of the computing device. Then, the method detects a first portion of gesture for selecting a first content item in the list of content items. While the first content item is selected, the method detects a trigger for a second portion of the gesture, and then selects at least one second content item based on the second portion of the gesture. The trigger for the second portion of the gesture is a scroll gesture, and while the first content item is selected the scrolling of the list is disabled. A method for manipulating multiple content items in a list of content items is also presented.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part application of US applicationhaving Ser. No. 14/027,186 filed on 14 Sep. 2013, the entirety of whichis incorporated herein by reference.

FIELD OF INVENTION

The present invention relates to gesture-based operations in electronicdevices with display, in particular gesture-based selection andmanipulation methods on a list of selectable content items displayed onsuch devices.

BACKGROUND OF INVENTION

Many programs in computing devices such as personal computers or smartphones feature editable content items displayed in the form of a list. Alot of times, selection of such items has to be done individually foreach item, such as by checking a checkbox or tapping on the item. Suchselection methods are prone to errors in both user side and machineside, and are not reliable and convenient to use, especially when alarge number of items are to be selected. Therefore a simpler and moreintuitive method for selecting multiple items at a time is desired toimprove the efficiency of the user interacting with the content items.

SUMMARY OF INVENTION

In light of the foregoing background, an aspect of the present inventionis a method for selecting multiple content items in a list of scrollablecontent items. In an embodiment, the method first displays a list ofcontent items scrollable along a first direction on a display of thecomputing device. Then, the method detects a first portion of gesturefor selecting a first content item in the list of content items. Whilethe first content item is selected, the method detects a trigger for asecond portion of the gesture, and then selects at least one secondcontent item based on a remainder of the second portion of the gesture.The trigger for the second portion of the gesture is a scroll gesture,and while the first content item is selected the scrolling of the listis disabled.

In an exemplary embodiment, the first portion of the gesture is a dragsubstantially orthogonal to the first direction, and in anotherexemplary embodiment the second portion of the gesture is a dragsubstantially parallel to the first direction. That means the firstportion of the drag is substantially orthogonal to the second portion ofthe gesture.

In another aspect of the present invention, a method of manipulating aplurality of content items in a list of content items is presented. Inan embodiment, the method first displays a list of content items along afirst direction on a display of a computing device. A plurality ofcontent items in the list of content items are selected, and eachcontent item has a parameter value in a parameter field. The method thendetects a multi-input gesture relative to the selected content items,and determines a direction and displacement of each input of themulti-input gesture. Based on the direction and displacement, theparameter values of the selected content item changes by an amount ofchange.

In different embodiments, the multi-input gesture can be a multiple draggesture, a spread gesture or a pinch gesture. In different embodiments,the parameter values of the selected content items can shift by a sameamount, or spread out or converge such that the parameter values of thecontent items changes in a way where a difference between adjacentcontent items increase or decrease across the selected content items.

This method of selection provides many advantages over the existingmethods. First, the entire selection operation, regardless of the numberof items to be selected, can be completed in one single motion of asingle finger. This method is much more convenient than existing methodswhere content items has to be selected individually using one gesturefor each content item. The selection is also intuitive to the user asthe user only needs to drag his finger over the desired content items toselect them.

The manipulating method of the present invention is also convenient forthe user to change multiple parameter values at one time, such as timesfor reminders. This is especially useful in situations where sometimesthe parameter values need to be changed relative to another contentitem.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a device where the method according tothe present invention can be implemented in.

FIG. 2 is a flow diagram of a method for selecting multiple contentitems according to an embodiment of the present invention.

FIGS. 3 a-3 c illustrates a mobile device in a method for selectingmultiple content items according to an embodiment of the presentinvention.

FIG. 4 is a flow diagram of a method for selecting and manipulatingmultiple content items according to an embodiment of the presentinvention.

FIG. 5 is a flow diagram of a method for manipulating multiple contentitems according to an embodiment of the present invention.

FIG. 6 a-6 b illustrates a mobile device in a method for manipulatingmultiple content items according to an embodiment of the presentinvention.

FIG. 7 is a flow diagram of a method for manipulating multiple contentitems according to an embodiment of the present invention.

FIG. 8 a-8 b illustrates a mobile device in another method formanipulating multiple content items according to an embodiment of thepresent invention.

FIG. 9 is a flow diagram of a method of manipulating multiple contentitems according to another embodiment of the present invention.

FIG. 10 a-10 b illustrates a mobile device in a method for manipulatingmultiple content items according to an embodiment of the presentinvention.

FIG. 11 is a flow diagram of a method of manipulating multiple contentitems according to another embodiment of the present invention.

FIG. 12 is a flow diagram of a method of selecting multiple contentitems in a multiple dimensional list, according to an embodiment of thepresent invention.

FIG. 13A-B illustrate a device during a method of selecting multiplecontent items in a two dimensional list, according to an embodiment ofthe present invention.

FIG. 13C illustrates the device in FIG. 13A applying an alternativeselection method, according to an embodiment of the present invention.

FIG. 13D-E illustrate a device during a method of selecting multiplecontent items in a three dimensional list, according to an embodiment ofthe present invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention are described below using acomputing device as example, such as one having the structure as shownin FIG. 1. The computing device (or device) 100 comprises a display 102,a processor 104, a memory 106, and a gesture module 108 that allows thedevice 100 to recognize gestures made by a user on its own or combiningwith other components. In an embodiment of the mobile device 100, thedisplay 102 is a touch-sensitive display or touchscreen that is capableof detecting a user touching the display 102 by a finger, a stylus pen,or other appropriate devices. The gesture module 108 then analyzes theinput to the display 102 to recognize the gesture made by the user. Thetouch-sensitive display can make use of resistive, capacitive, acousticwave, infra-red, optical imaging or other technologies. In otherembodiments, the gesture module 108 is connected to a sensing module notintegrated into the display 102 for non-touch based gesture recognition.

In one embodiment, the processor 104 executes computer-readableinstructions stored in memory 106, such as instructions that cause themethod of the present invention to be performed. In differentembodiments, the computing device 100 also comprises I/O connectors 110for connecting to external devices such as an external sensing module,one or more cameras 112 for capturing images, a wireless communicationmodule 114 for communication with other computing devices or a server,an audio output module 116 for providing audio output, etc.

A first aspect of the present invention is a method for selectingmultiple content items in a list of content items. An embodiment of sucha method is illustrated in the flow chart as shown in FIG. 2. In step200, the device displays a list of content items along a first directionsuch as vertical or horizontal. The content items could be text items,numerical expressions, audio clips, video clips, any other forms ofcontent or a combination thereof as the program supports. Each contentitem is selectable by the user either individually or as a group.

In an exemplary embodiment, the list of content items is scrollablealong the first direction, such as vertical (i.e. up-down). A scrollgesture or a scroll command, such as a single drag in the verticaldirection, can be detected by the device to trigger the scrolloperation, resulting in some previously undisplayed content items to bedisplayed on the display screen for example. In a further embodiment,the list of content items is scrollable only along the first directionbut not in other directions.

In step 202, the device detects a first portion of a gesture by the userselecting a first content item in the list. For a touch-sensitivedisplay as described above, the gesture is detected by sensing a fingercontact on a location corresponding to the first content item, usuallyright on top of the first content item, or in specifically designatedareas on the display screen. For other devices, the location can be alocation proximate the first content item, or the finger can be at apoint in space that is determined to be pointing at the correspondinglocation.

In an exemplary embodiment, the first portion of the gesture is asingle-finger drag initiated on the first content item and runningsubstantially orthogonal to the first direction. This configurationminimizes the possibility that the select operation being mistaken as ascroll operation, which is a single-finger drag along or substantiallyparallel to the first direction. In an exemplary embodiment, thisgesture is identical to the gesture for selecting a single content item,which means that only the first content item is selected if the gestureis concluded after the first portion, which means that the gesture forselecting multiple content items is an extension of the gesture forselecting a single content item. This feature makes the selectionprocess more intuitive to the user and simplifies the operation of theprogram.

In various other embodiments, the first portion of the gesture can bemulti-finger drag running substantially orthogonal to the firstdirection, or a single tap, a double tap, a tap-and-hold etc, so long asthe first portion of the gesture can be distinguished from a scrollgesture. However, it is more preferred that the select operation isquick and can be performed using a single finger so as to be easilyperformed while holding the computing device in a hand.

In step 204, the device selects the first content item in response todetecting the first portion of the gesture. In an exemplary embodiment,a buffer is set by the device such that the first content item is deemedselected only if the drag distance or displacement exceeds the buffer.The buffer can be set at around a quarter of the width of the screen ora fixed number of pixels. In various embodiments, the selection of thefirst content item is indicated to the user by highlighting the selectedcontent item using another color in all or part of the content item, orby shifting the selected content item relative to unselected contentitems, or by playing a sound effect, or by vibration of the device, ordisplaying a check mark, or a combination of the above.

While the first content item is selected, in step 206, the devicedetects a trigger for the second portion of the gesture. It is notedthat the finger is not released from the display between the firstportion and the second portion of the gesture, meaning that the deviceregards the two portions of the gesture as a single gesture but notseparate gestures, even different portions of the gesture may instructthe device to perform multiple operations. In different embodiments, thedevice may allow or deny a pause between the first and second portions.

In an exemplary embodiment, the trigger for the second portion of thegesture is identical to a scroll gesture or scroll command along thefirst direction. In one embodiment, the scroll gesture along the firstdirection is determined as a drag where the component along the firstdirection is larger than or equal to the component orthogonal to thefirst direction. While the first content item is selected and thegesture is not yet completed or concluded, scrolling of the list alongthe first direction is disabled so the list of content items will stayat the original location, allowing the finger to move over and select atleast one second content item, as explained in more detail below.

In step 208, the device selects the at least one second content itembased on the second portion of the gesture. In one embodiment, thedevice first determines whether a next or adjacent second content itemin the direction of the second portion of the gesture is present. Ifpresent, then the next second content item is determined to be selectedwhen the device determines that the finger moves over to the next secondcontent item. In different embodiments, whether the finger has movedover to the next second content item can be determined by the distancemoved along the first direction, or by comparing the present location ofthe finger and the area occupied by each second content item. Forexample, when the distance moved is approximately the same as the heightof a content item, it is very likely that the finger has moved from onecontent item to the next content item. Alternatively, when the fingercrosses a certain point on the display that is occupied by one contentitem, the device can also deduce that the finger moves over to the nextcontent item. In one embodiment, a buffer is also employed for theselection of the next second content item. For example, the buffer canbe set such as the point of contact is a certain pixels over theboundary of the content item so as to prevent accidental selection ofthe next item.

In an alternative embodiment, a matrix or table of content items isscrollable in more than one direction, and scrolling in all directionsare disabled during a gesture once the first content item is selected.The selection of the first content item can be by a double tap or othergestures that can be distinguished from a scroll gesture for any of thedirections.

In another alternative embodiment, the scrolling of the list of contentitems is disabled when the device detects that the first content item isselected, but is reenabled as the second portion of the gestures reachesan edge of the display screen. This allows the user to select morecontent items than those displayed in a single gesture.

In an exemplary embodiment, a specific second content item is selectedimmediately when the second portion of the gesture is determined to moveover the content item. In another embodiment, the device detects thegesture is concluded before determining the ending location of thefinger, and then selects all content items between the first contentitem and the one at the ending location simultaneously. In both cases, acontinuous sublist of content items is selected by the operation.

FIG. 3 a-3 c shows an implementation of the above method in a schedulingapp. In FIG. 3 a, which is the state before the method starts, device300 is shown as a smart phone with a touch screen display 302. A list offive content items 304 a-304 e is displayed along a vertical direction.Each content item 304 a-304 e comprises a task description (e.g. Task 1)and a time 306 a-306 e associated to the task description, which areboth editable by a user. The content items 304 a-304 e in the list canbe sorted in a chronological order based on the times 306 associated tothe task descriptions. A menu bar 308 is displayed at the bottom of thedisplay 302, with a date 310 of the tasks displayed at the centerthereof. A left button 312 and a right button 314 are also displayed atthe two sides of the date 310 to allow the user to switch to tasks foranother date. In one embodiment, an option button (not shown) isdisplayed instead of the left button 312 to allow the user to configurethe settings of the app.

In an exemplary embodiment, the menu bar 308 is a pull-up menu where acalendar is displayed when pulled up. The calendar allows the user todirectly switch to a particular date to edit or check the tasks on thatdate.

In FIG. 3 b, the device 300 detects a user selecting a first contentitem 304 a by detecting a first portion of a drag, denoted by arrow 316,at the first content item 304 a in a direction substantially orthogonalto the direction of the list, which in this case is horizontal. Thedirection of drag is shown as left in this figure, but it is understoodthat a drag to the right can achieve the same effect. Once the distanceof the drag exceeds the buffer, which for example can be around aquarter to a half of the width of the screen, the first content item 304a is selected, which can be indicated to the user using visualindications and/or audio indications as mentioned above. In thisembodiment, the first content item 304 a follows the movement of thefinger so the first content item 304 a will shift to the left when thedrag is to the left.

The device 300 then detects the trigger for the second portion of thegesture, which is a drag that runs downwards, denoted by arrow 318, andtravelling over content items 304 b, 304 c and 304 d. When the dragmoves overs a content item, the device 300 selects the content itemalong with the first content item 304 a. In an exemplary embodiment, theindication of a content item being selected during the second portion ofthe gesture is identical to that during the first portion of thegesture. That means content items 304 b, 304 c and 304 d are highlightedwith the same color and shifted in the same direction as the firstcontent item 304 a. Once the finger moves over to another content item,the previously selected content item will return to its originalposition. It can be seen in FIG. 3B that content item 304 b is alreadyfully returned to its original position while content item 304 c isstill returning to its original position. Content item 304 d shows thatthe currently selected content item is shifted to the same direction asthe first portion of the gesture.

In another embodiment, while the visual indication is the same for allcontent items, different audio indications can be different for eachcontent item selected. In a specific example, a pitch of the audioindication can change for each content item selected along the list. Thepitch can be set such that it follows a major chord or a specificsequence, behaving as if each content item is a key for a musicalinstrument like a piano.

While the drag is continuing, the information displayed in the menu bar308 changes. The date 310 is still displayed at the center of the menubar 308, but at the left of the menu bar 308 is a box 320 showing thenumber of content items selected. In this embodiment, three contentitems (304 a, 304 b and 304 c) are deemed as selected, while theselection of content item 304 d will only be reflected either when thefinger moves to the next content item 304 e, or releases from thedisplay. After the selection gesture is concluded, tapping this box 320will unselect all selected content items simultaneously. A recycle binicon 322 is also displayed at the right side of the date 310. After thedragging is concluded and the selected content items are finalized, theselected content items can be deleted by the user interacting with therecycle bin icon 322, such as tapping the recycle bin icon 322.

Referring to FIG. 3C, it can be seen that all five content items 304a-304 e are selected, as indicated by the number “5” shown in the box320 in the left hand side of the menu bar 308.

In another embodiment as shown in FIG. 4, the device further detects athird portion of the gesture. Steps 400-408 are identical to steps200-208 as described above. In step 410, the device detects a thirdportion of the gesture for performing an operation on the selectedcontent items. In an exemplary embodiment, the second portion of thegesture is entirely a drag substantially parallel to the first directionso that the device can distinguish the third portion of the gesture fromthe second portion of the gesture by their directionalities. In afurther embodiment, the third portion of the gesture has a samedirectionality than the first portion of the gesture.

Using the example in FIG. 3 above, the third portion of the gesture canbe a dragging gesture along the horizontal direction in an embodiment,which is substantially orthogonal to the direction of the list. Thethird portion can be the in the same or opposite direction as the firstportion, and the operation can be deletion of the content items (same asthe function of the recycle bin icon 322), highlighting the contentitems for indicating different levels of importance or priority,grouping the content items together for group editing purposes, or otheroperations. In another embodiment, the third portion of the gesture is adragging gesture that starts in the horizontal direction but has acomplex trajectory like a circle, a star or a Z-shape, etc. The start inthe horizontal direction is needed for distinguishing the third portionfrom the second portion, and the complex trajectory can increase thenumber of operations a user can do to the content items using a singlemotion.

In another embodiment, the third portion of the gesture involves anotherfinger than the one used in the first portion and the second portion ofthe gesture. For example, when the first portion and the second portionof the gesture is a dragging gesture using a first finger, the thirdportion of the gesture can be a tap using a second finger, while thefirst finger is still held at the position of the second content item sothat the device detects that the gesture is still not concluded.

If the content items are arranged in a way such that they are completelyaligned, such as the arrangement in FIG. 3 a, the second portion of thegesture would result in a continuous sublist of content items beingselected. In an alternative embodiment, the list of content items has anoffset in their horizontal locations, such that the finger can move oversome of the content items without crossing over others, allowing theuser to select multiple content items that are not necessarily adjacentto each other in one single gesture. In another embodiment, after thecontent items are selected, individual selected content items can bedeselected using operations similar to selecting an individual contentitem in order to have the desired content items be selected only.Multiple content items can also be deselected by following the samesteps as explained above for selecting the content items.

In another embodiment, while the second portion of the gesture is inprogress, backtracking of the gesture, or in other words, having thegesture move in an opposite direction to the direction of selectionwould result in unselection of a selected content item. For example, ifthe second portion of the gesture is a downward drag, then a upward dragover a content item would unselect the content item.

FIG. 5 shows a method of editing or manipulating the content items afterthey are selected, according to an embodiment of the present invention.In step 500, the device displays a list of content items on its displayalong a first direction, with a plurality of content items in the listbeing selected. Each content item has a parameter field having aparameter value associated thereto, where the parameter values of thecontent items are sortable and can be manipulated using alphanumericaloperations. Examples of the parameter field include numerals (e.g. 1, 2,3 . . . ), alphabets (e.g. A, b, c . . . or i, ii, iii . . . ), time(9:00, 10:00, 11:00 . . . ) or other parameter types. The selectedcontent items can be in a continuous list or can be discrete contentitems. The method of selection can be a method as described above or aconventional selection method, as long as the appropriate content itemsare selected.

In step 502, the device detects a multi-input gesture being initiatedrelative to the selected content items. For the purposes of thisinvention, a multi-input gesture is a gesture where the movements ofmore than one input is interpreted together by the device as a singlegesture. In various embodiments, the multi-input gesture can beinitiated anywhere on the selected content items, on a specific selectedcontent item among the selected content items, on the time parameter ofthe content items, or anywhere on the display screen. The location ofinitiation can be determined by the location of one of the fingers, or amid-point of the multiple fingers.

In step 504, the device determines a direction of movement and adisplacement of each input of the multi-input gesture. In differentembodiments, the directions of movement of the multiple inputs can besubstantially parallel, substantially opposite, extending radiallyoutwards or inwards, or rotating in a clockwise or counterclockwisedirection. The displacement can be determined by a total distance moved,a net distance moved along a specified direction such as along the firstdirection, or an angle of rotation.

In step 506, the device changes the parameter values of the contentitems by an amount of change based on the direction and displacement ofeach input of the multi-input gesture. In an exemplary embodiment, thedirection of each input determines the operation type and therefore howthe parameter values should be changed, such as which expression shouldbe used to calculate the amounts of change. In another exemplaryembodiment, the magnitude of the amount of change is generallyproportional to the displacement of the inputs.

FIGS. 6 and 7 show a specific example of the method described above,according to an embodiment of the present invention. In step 700, thedevice displays a list of content items along the first direction, witha plurality of content items 604 a-604 e selected as shown in FIG. 6 a.Each content item 604 a-604 e has a task description and also aparameter field having a parameter value 606 a-606 e. In this specificembodiment, the parameter is shown as a time. As shown in FIG. 6A, thetimes 606 a-606 e are all displayed as 9:00 am. The menu bar 608 has abox 620 for indicating the number of content items selected and arecycle box icon 622 for deleting the content items. As indicated by thenumber “5” in the box 620, all five content items are selected in FIG. 6a.

In step 702, the device detects a multi-finger drag gesture along thefirst direction relative to the selected content items. A multi-fingerdrag gesture is defined as more than one finger dragging in asubstantially same direction. In step 704, a direction and adisplacement of each finger of the multi-finger drag gesture isdetermined. When the first direction is vertical, the direction of thedrag can be up or down, as shown by the arrow 616 in FIG. 6 b. Thedisplacement can be determined by an average displacement of the fingerseither upwards or downwards.

In step 706, based on the direction and the displacement of each inputof the multi-finger drag gesture, the device shifts the times 606 a-606e by a same amount. When the drag is downwards, the parameter willincrease in value, or in this case, the time is delayed. As shown inFIG. 6B, the times 606 a-606 e (9:15 a.m.) are shifted from the times inFIG. 6A (9:00 a.m) by 15 minutes. On the other hand, when the drag isupwards, the times will shift forward. The relationship of thedirections of drag and time shift can be changed according to userneeds.

In an exemplary embodiment, the amount of change is generally directlyproportional to the displacement. That means doubling the displacementwould also double the amount of change of the parameter values and viceversa. In another exemplary embodiment, the parameter value 606 a-606 eincrements or decrements in predetermined intervals. For example, inFIG. 6 the interval is set as 5 minutes, 15 minutes, or 1 hour. Invarious embodiments, the interval is adjustable by a user by a gestureor by configuring the settings before the gesture. Assuming the intervalin this example is 15 minutes, FIG. 6B shows that the times 606 a-606 eis shifted by one interval. If the drag continues and the displacementcontinues to increase, the times 606 a-606 e will then change to 9:30a.m., then 9:45 a.m, until the gesture is completed.

In an alternative embodiment, the amount of change of the parametervalues are further dependent on a hold duration of the gesture. Byholding the gesture for a period of time, the amount of change can beincreased even if space is limited therefore preventing the displacementto further increase. In a further embodiment, the amount of change onlystart depending on the hold duration when the fingers already reach theedge of the display.

FIGS. 8 and 9 describe another method of manipulating the content itemsaccording to an embodiment of the present invention. In step 900, thedevice 800 displays a list of content items along the first direction,with a plurality of content items 604 a-604 e selected as shown in FIG.8A, same as that in FIG. 6A of the previous example. In step 902, thedevice detects a spread gesture along the first direction relative tothe selected content items. Then, in step 904, the device sets one ofthe selected content items as a reference content item, and theparameter value of the reference content item becomes a referenceparameter value. In different embodiments, the reference content item isdetermined by the location of initiation of the gesture, such as by oneof the fingers or the mid-point of the fingers. In another embodiment,the reference content item is determined as the first or topmost contentitem among the selected content items (i.e. 804 a), and independent tothe location of initiation of the gesture, or it can be the last or mostrecently selected content item.

In step 906, the device determines the displacement of the inputs alongthe first direction, similar to the example above. In step 908, thedevice spreads out the parameter values of the selected content items806 b-806 e from the reference parameter value 806 a, which means thatthe parameter values 806 b-806 e moves away from the reference parametervalue 806 a and the amount of change of the parameter value for eachselected content item increases from the last content item. In anexemplary embodiment, the reference parameter value 806 a staysunchanged, and the parameter value of the selected content items changesin a way that a difference between adjacent content items increasesacross the selected content items. As seen in FIG. 8B, the referenceparameter value 806 a stays at 1:00 pm. The parameter values 806 b-806 eis changed such that there is a difference of 45 minutes betweenadjacent content items, i.e. 1:45 pm, 2:30 pm, 3:15 pm and 4:00 pm. Ifthe displacement of the drag increases such that the difference isdetermined to be 1 hour, the parameter values 806 b-806 e will become2:00 pm, 3:00 pm, 4:00 pm and 5:00 pm.

In an alternative embodiment, the reference content item is determinedby the location of initiation, such as 804 d in FIG. 8 a. In such case,upon detecting the spread gesture, the time 804 e will be pushed backfrom time 804 d (e.g. from 1:00 p.m. To 1:45 p.m.), but times 804 a-804c will be pushed forward from time 804 d (e.g. from 1:00 p.m. To 12:15p.m. Etc.)

Another method of manipulating content items according to the presentinvention is described in FIGS. 10 and 11. In this method, the gestureis a pinch gesture, and the change of parameter value is basically areverse of the above embodiment. The parameter values of the selectedcontent items will converge towards the parameter value of the referencecontent item, as seen in FIG. 10 b.

Another embodiment of the present invention for selecting multiplecontent items in a list of content items is shown in FIGS. 12 to 13. Instep 1200, device 1300 displays a scrollable list of content items 1304on its display 1302. In this embodiment, the list of content items is amultidimensional list instead of a one dimensional list above, meaningthat content items 1304 are arranged on the display 1102 in more thanone direction. For example, in the embodiment as shown in FIGS. 13A-Cthe content items are arranged in a tile or grid format, where thecontent items are arranged in both vertical and horizontal directions,i.e. rows and columns. In the embodiment as shown in FIGS. 13D-E thecontent items are arranged in a three dimensional format with a depthinformation in addition to the vertical and horizontal directions.

The list of content items is scrollable in at least a first direction,generally along one of the directions where the content items arearranged. For example, if the content items are arranged in a gridformat, the list of content items can be scrollable along the verticaldirection, or along the horizontal direction. The list can be made to bescrollable in only the first direction or in all directions where thecontent items are arranged along. Alternatively, the list of contentitems is freely scrollable along an arbitrary direction regardless ofthe direction of arrangement.

In FIG. 13A, the list of content items comprises a gap between adjacentcontent items, but content items can also be placed immediately adjacentto each other without a presence of the gap. In the case where a gap ispresent, the gap can display thereon unselectable content such as awallpaper or information related to a content item such as type andsize, or date of creation or modified, or any attributes that are beingused to sort the content items. The gap can also display thereon otherselectable content such as a content item which belongs to the same listbut does not belong to the current page or layer of content items.Rather, the content item belongs to another layer of content items whichis selectable with a content item in the current layer with a singlegesture, as explained in more detail in the next embodiment.

In the embodiment as shown in FIGS. 13D-13E, the content items arefurther layered in the front-back direction or z-axis addition to beingin tile or grid format, meaning that they are arranged in threedimensions. In this embodiment a gap is present between adjacent contentitems in the same layer, and content items in the next layer is at leastpartially exposed for selection at the gaps. As shown in the figure, thesize of each content item does not need to be the same across differentlayers, or even within the same layer.

For explanation purposes, the first content item 1304 a is defined asthe top left content item on the display 1302, as is the same as inprevious embodiments. In step 1202, the device 1300 detects a firstportion of a gesture selecting the first content item 1304 a. Since thelist of content items are arranged in multiple directions, the selectionof the first content item 1304 a can be made by detecting anon-directional input such as tapping, double tapping or tapping andholding the first content item 1304 a. The first content item 1304 a canalso be selected by detecting a initiation of a drag at the firstcontent item 1304 a, with the drag running at a direction that is notscrollable before the selection gesture. It is understood to one skilledin the art that selection methods are not limited to physicalinteraction between the user to the content item, and can also be basedon optical or chemical means such as facial recognition, neural synapsisdetection, etc.

In step 1204 the device 1300 selects the first content item 1304 a inresponse to the first portion of the gesture, and at the same time thedevice 1300 disables scrolling of the list of content items along thefirst direction. When the list of content items is scrollable along morethan one direction, the device 1300 can either disable the scroll in alldirections or leave the scroll enabled in directions other than thefirst direction.

In step 1206, the device 1300 then detects a trigger for a secondportion of the gesture. In one embodiment, where the first portion ofthe gesture is a non-directional gesture, the trigger is a drag gestureaway from the first content item. The drag can be along one of thedirections where the content items are arranged in, or in any directionalong the plane of content items. In another embodiment, the drag has afirst component along the first direction, where the list of contentitems is scrollable along if no content items are selected. Thescrolling along the first direction is disabled upon the first contentitem being selected, allowing other content items to be selected withthe same gesture when the gesture moves from the first content item toan end point. Furthermore, the scrolling along the first direction canbe re-enabled upon detecting the gesture reaching an edge of the screenalong the first direction.

In step 1208, the device 1300 selects at least one second content itemsin response to the second portion of the gesture. For explanationpurposes, the second portion of the gesture is a drag in a straightline, ending at content item 1304 h. In the embodiment as shown in FIG.13B, the content items in the list are first arranged by an item indexthereof, first in rows and then columns. The content items are labeledfrom 1304 a to 1304 h to represent the order of item index. The itemindex can be a sortable attribute of the content item as mentionedabove. When the gesture ends at content item 1304 h, the content itemswith item index between that of first content item 1304 a and contentitem 1304 h are all selected. It is obvious that the content items canbe arranged first in columns and then rows, and the way the contentitems are arranged is not necessarily dependent to the first direction.

In the embodiment of FIG. 13C, the content items are selected based onthe coordinates of the content items on the display. The content itemsthat are within the rectangular area defined by the vertical andhorizontal displacement of the drag gesture are selected. As shown inFIG. 13C, the content items 1304 a, 1304 b, 1304 d, 1304 e, 1304 g and1304 h are selected as the vertical displacement is two rows and thehorizontal displacement is one column, but content items 1304 c and 1304f are not selected.

Alternatively, the content items can be selected by identifying thecontent items that the drag gesture passes through along the route ofthe drag. For example, if the drag gesture first moves from firstcontent item 1304 a downwards to content item 1304 g then to the rightto content item 1304 h, only content items 1304 a, 1304 d, 1304 g and1304 h will be selected. This allows a greater freedom for the selectionto only select necessary content items comparing with aforementionedindex or area selection methods.

In the embodiment as shown in FIG. 13E, only two layers of content itemsin the list are shown, referred herein as the top layer and the bottomlayer respectively. The content items in the top layer are entirelydisplayed, while the content items in the bottom layer is partiallyunderlayed to a corresponding content item in the top layer.

The selection gesture starts from the first content item 1304 a at thetop layer. The first content item 1304 a can be selected using similarmeans as mentioned above. The corresponding content item at the bottomlayer is labeled as 1306 a. The second portion of the gesture then movesfrom the first content item 1304 a to content item 1306 h at the bottomlayer, which corresponds to content item 1304 h at the top layer. Inthis embodiment, the content items are selected based on area selection,and content items 1304 a, 1304 b, 1304 d, 1304 e, 1304 g and 1304 h,along with their corresponding content items at the bottom layer, areselected by this gesture. In different embodiments, the second portionof the gesture can move directly onto the content item 1306 h at thebottom layer, or first moving to the content item 1304 h at the toplayer and then changing the layer to the bottom layer to end at contentitem 1306 h. The changing of layer can be detected by a continued holdon a content item, an increase of contact area or pressure at thecontent item, or tapping another button on the display etc.

In an alternative embodiment, instead of partially overlaying thecontent items in different layers, the device can change thetransparency of the content items in different layers in order to showthat multiple layers are present in the list. This embodiment caneliminate the need of the presence of a gap between adjacent contentitems on the same layer in order for the content items in the subsequentlayers to be selected.

In one embodiment, when the second portion of the gesture reaches anedge of the display screen in a scrollable direction, scrolling of thelist of content items is re-enabled for additional content items to bedisplayed. In another embodiment, the size of the displayed contentitems is decreased when the second portion of the gesture reaches theedge, therefore allowing more content items to be displayed forselection. The size decrease does not need to be uniform across thedisplay or by the same ratio along different directions. For example thecontent items farther away from the current position of the finger canbe compressed more than the content items close to the current positionof the finger, and content items can be compressed more along a longeredge of the display then along a shorter edge of the display.

In an alternative embodiment of the present invention, thethree-dimensional list is made up of stacks of content items instead oflayers of content items. Each stack of content items can comprise of adifferent number of content items, and the stacks of content items canthen be arranged in rows or columns or grid formats comprising the firstdirection. A different number of content items can be selected for eachindividual stack, for example based on the length of tap detected by thedevice on each stack. A drag gesture from one stack to another stackwould result in selection of a same number of content items selected formultiple stacks within the area or route or index of selection.

In another embodiment of the present invention, the device is capable ofdisplaying information and detecting a gesture in three dimensionalspace. For example, the device can comprise a three dimensionalprojection system that projects content items on a three dimensionalspace, and also comprise a sensing system that senses a specific body,such as a pointer or a finger, in the three dimensional space. Thedevice can be a virtual reality device that presents the content itemsin a virtual space, and having a sensing system that maps the pointer orfinger into a location in the virtual space.

The selection method would be similar as in the above embodiments, wherescrolling of the list of content items along an initially scrollablefirst direction is disabled upon a first content item being selected.The first portion of the selecting gesture can be a non-directionalgesture such as a tap, and the second portion can be a straight linefrom the first content item to an end point with displacements along allthree axes. The embodiments of the present invention are fully describedabove. Although the invention is illustrated in specific embodiments,one skilled in the art would understand that variations andmodifications can be made in the details of the implementation withoutdeparting from the scope and spirit of the invention. The scope ofinvention will be construed by the claims below.

For example, the interval can be adjusted by the user. In oneembodiment, the gesture for manipulating the parameter values ispreceded by a drag orthogonal to the first direction. The distance ofthe orthogonal drag can be used to determine the interval in the gesturethat follows. The current selected interval can be displayed to the useras the drag is in progress.

In another embodiment, the interval is dependent on the number of inputsof the multi-input gesture. For example, a two-finger gesture willresult in a smaller interval than a three-finger gesture.

In one embodiment, a limit is set for the parameter values so that whenthe parameter values are changed using a method described above, theparameter values cannot move over the limit. For example, limits can beset at the beginning of the day and the end of the day, or in pinchgestures, the time of the reference content item.

In one embodiment, the time shift operation can be operated with oneinput. In an exemplary embodiment, the method detects a single input ata specific area of the content item, for example on the time of thecontent item. The movement of the finger along the first direction willshift the time of all selected content items by a same amount. In afurther embodiment, the interval of the time shift can be adjusted bythe user by detecting the finger dragging orthogonal to the firstdirection by a predetermined distance before moving along the firstdirection.

What is claimed is:
 1. A method comprising: displaying a list of contentitems scrollable along a first direction on a display of a computingdevice, said list of content items comprising a plurality of contentitems arranged in said first direction and at least one seconddirection; detecting a first portion of a gesture selecting a firstcontent item in said list of content items; while said first contentitem is selected, detecting a trigger for a second portion of saidgesture; and selecting at least one second content item in said list ofcontent items in response to said second portion of said gesture;wherein said trigger for said second portion of said gesture comprises afirst component being a scroll gesture along said first direction, andwherein scrolling of said list of content items along said firstdirection is disabled upon said first content item being selected. 2.The method according to claim 1, wherein said first portion of saidgesture is a drag in a direction substantially orthogonal to said firstdirection.
 3. The method according to claim 1, wherein said selectingstep comprises the step of detecting said second portion of said gesturemoving over said at least one second content item.
 4. The methodaccording to claim 1, wherein said at least one second content item isselected based on an area defined by displacements of said secondportion of said gesture along said first direction and said at least onesecond direction.
 5. The method according to claim 1, wherein said atleast one second content item is selected based on an item indexthereof, said item index being a sortable attribute to said list ofcontent items.
 6. The method according to claim 1, wherein said list ofcontent items comprises a plurality of layers of content items with eachlayer of content item comprising a plurality of content items arrangedin said first direction and said at least one second direction, whereinsaid first content item is in a different layer than at least one ofsaid at least one second content items.
 7. The method according to claim6, further comprising the step of detecting a change of layer ofselection during said second portion of said gesture.
 8. The methodaccording to claim 7, wherein said step of detecting a change of layerof selection comprises detecting contact to a content item outside saidlayer of selection, where said content item outside said layer ofselection is at least partially displayed and linked to another contentitem within said layer of selection.
 9. The method according to claim 1,wherein said list of content items comprises a plurality of stacks ofcontent items arranged in at least said first direction, each said stackof content items comprising at least one content item.
 10. The methodaccording to claim 1, further comprising the step of: while said firstcontent item and said at least one second content item are selected,detecting a third portion of said gesture for performing a predeterminedoperation on said first content item and at least one second contentitem.